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ABSTRACT 


Water stress greatly determines plant yield as it affects plant metabolism, 
photosynthesis rate, chlorophyll content index, number of leaves, 
physiological, biochemical compound, and vegetative growth. The research 
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aimed to detect and classify water stress of cultured Sunagoke moss into 
several categories i.e. dry, semi-dry, wet, and soak by using a low-cost 
commercial visible light camera combined with a deep learning model. 
Cultured Sunagoke moss is a commercial product which has the potential use 
as rooftop-greening and wall-greening material. This research compared the 
performance of four convolutional neural network models, such as 
SqueezeNet, GoogLeNet, ResNet50, and AlexNet. The best convolutional 
neural network model according to the training and validation result was 
ResNet50 with RMSProp optimizer, 30 epoch, and 128 mini-batch size; this 
also gained an accuracy rate at 87.50%. However, the best result of the 
convolutional neural network model on data testing using confusion matrices 
on different data sample was ResNet50 with Adam optimizer, 30 epoch, 128 
mini-batch size, and average testing accuracy of 94.15%. It can be concluded 
that based on the overall results, convolutional neural network model seems 
promising as a smart irrigation system that real-time, non-destructive, rapid, 
and precise method when controlling water stress of plants. 
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1. INTRODUCTION 

Environmental conditions such as temperature, humidity, carbon dioxide, light intensity, water, and 
nutrient solution greatly affect plants growth [1]. Optimum environmental conditions can increase plant 
growth and productivity. However, if one of those environmental factors is not optimum, then it leads to 
plant stress. Water stress comes into a major concern among various types of stress in plants due to its 
implication [2]. It can affect plants metabolism and further influence plant growth and development which 
also can reduce plants productivity [3]. When water stress in plants occurs, the chemical signals are 
transmitted from root to leaf through the xylem which can stimulate closure of stomata and lower 
photosynthesis performance. Maseko et al. [4] has investigated water stress on several plants (Amaranthus 
cruentus L., Corchorus olitorius L, and Vigna unguiculata (L.) Walp and Beta vulgaris L.) affected plant 
height, chlorophyll content index, yield, and the number of leaves. Moreover, Hendrawan and Murase [5] 
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proved that water stress in moss plants corresponded to its photosynthesis rate. Water stress defines as a 
condition where plants experience either lack or excess of water than its requirement. Zhang et al. [6] 
observed that water stress affected root biomass and secondary metabolites on the Stellaria dichotoma L.var. 
lanceolata Bge. Ibrahim and Abdellatif [7] 1n their research concluded that water stress in wheat plants 
(Triticum aestivum L. cv. Giza 168) reacted on the biochemical components such as phenolic compounds, 
flavonoids, amino acids, sugars, total soluble sugars, protein, and more. Furthermore, it has been shown that 
water stress affected the antimicrobial activity of some medicinal plants [8]. And more, water stress 
contributed to the vegetative growth, yield, and quality of potato plants [9]. Sun et al. [10] in his research 
found evidence that water stress affected the relative growth rate of some types of marsh herbaceous plants. 
Jiang et al. [11] also proved that it influenced both linearly and non-linearly on plants' height and elongation 
rate of winter wheat. Ju et al. [12] revealed the effect of water stress on the physiological, 
micro-morphological, and metabolomics on the grape vines. According to the water stress impacts on plants 
mentioned, it is urgent to develop methods that can detect water stress in plants accurately for assuring 
optimal plant growth. The water stress detection method in plants is determined by two majors: 
instrumentation (sensor) and modeling method. Rapid, real-time, and non-destructive sensing are principal in 
detecting water stress in plants. 

Water stress detection methods in plants have been widely developed such as spectral reflectance of 
plant leaves using spectrometer [13]. In research conducted by Ihuoma and Madramootoo [14], the spectral 
reflectance using fiber-optic spectrometer ranging from 200 nm to1150 nm can detect water stress in potato 
plants. Liu et al. [15] used a thermal camera to detect water stress in Eucalyptus microcarpa and Acacia 
pycnantha plants according to thermal indices of the leaf canopy. Khorsandi et al. [16] has also been 
successfully detected water stress in sesame plants using infrared thermography. Moreover, previous studies 
have widely used visible near-infrared (VIS/NIR) spectroscopy to detect water stress in plants [17], as 
research conducted by Xia [18], she used VIS/NIR to detect it in tomato plants. Furthermore, 
Fonseca et al. [19] was successful to detect water stress in blueberry plants non-destructively using VIS/NIR 
with the highest R? 0.69. Terahertz spectroscopy also became alternatives for non-destructive sensing to 
detect water stress in plants as carried out by Li [20]. She has successfully measured leaf water content in 
soybean plants using terahertz time-domain spectroscopy. Another non-destructive method that has been 
developed to detect water stress in plants is by using laser-induced fluorescence technology. This method was 
successfully applied in Arabidopsis plants [21]. Hyperspectral machine vision was investigated and proven 
effective to detect water stress in tomato plants [22]. A simpler and cheaper machine vision method using a 
low-cost commercial visible-light camera was also researched and proven effective to detect water stress in 
plants in a rapid and non-destructive way [23]-[25 ]. 

Artificial intelligence utilization as a model in predicting water stress on plants has been widely 
carried out by researchers. Wakamori et al. [26] used a multimodal neural network with a clustering-based 
drop (C-drop) to predict water stress in plants accurately with the highest R? 0.429. This model can be used 
for decision making for an irrigation system and proven precise and stable to detect water stress in plants. 
Kaneda et al. [27] used multi-modal sliding window-based support vector regression to predict water stress in 
plants with prediction effectiveness showed by the highest R-value accounting for 0.509. The least-squares 
vectors machine was investigated and proven effective for classifying some water stress categories in wheat 
plants [28]. Hendrawan and Murase [29] inspected artificial neural network (ANN) effectiveness to predict 
water stress in moss plants root mean square error (RMSE) with its validation accuracy of 0.0107. Although 
only a few researches on deep learning propose this will bring many possibilities for future research in 
detecting stress in plants [30]. As Anami et al. [31], who proposed the method to classify water stress in 
paddy crop resulting in training data accuracy of 92.89%. Other researchers using deep learning to detect 
stress on nitrogen deficiency in sorghum plants with an accuracy of 92% [32] and convolutional neural 
network (CNN) use to detect light stress in lettuce with an accuracy of 87.95% [33]. This kind of deep 
learning, CNN, showed an outstanding performance in image recognition scope and therefore this may bring 
potential methods to detect water stress in plants compared to other machine learning methods. A low-cost 
commercial visible light camera combined with deep learning will create a robust alternative method as it 1s 
rapid, real-time, non-destructive, accurate, and affordable. Such a combination has not been widely 
developed in research. 

The research aims to detect and classify water stress in the cultured Sunagoke moss Rhacomitrium 
japonicum into several categories (dry, semi-dry, wet, and soak) by using a low-cost commercial visible light 
camera tool and deep learning model. The result of the method developed can later be utilized to optimize 
irrigation control in the production of cultured Sunagoke moss in plant factory (closed bio-production 
system) so it can grow more optimal. Sunagoke moss which under water stress conditions i.e. lack of water 
(dry and semi-dry) or excess of water (soak) cannot photosynthesize optimally and certainly affect the yield. 
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Cultured Sunagoke moss is a very potential commercial product to use as rooftop-greening and wall-greening 
material. 


2. RESEARCH METHOD 

Sunagoke moss sample (500x500 mm, M-300, VARORE Co., Japan) consisting of several water 
stress conditions, dry (0-1 g g!), semi-dry (1-2 g gt), wet (2-3 g g'), and soak (>3 g g1) were set where g g`! 
is water weight (g) per initial dry weight (g). Figure 1 shows the moss sample. The water content of moss 
samples was set by initial dry weight. All moss samples were put on the growth chamber (Biotron NK 350, 
Japan) at optimal environmental condition, air temperature-15 °C, RH=80%, CO5-400 ppm, light 
intensity=86.5 mol m?s', and light duration=12 h. Each moss sample was given water to 4 g g' and allowed 
to dry to reach its initial dry weight. Generally, moss grows optimally at water content 2-3 g g! (wet 
condition). The sample moss image was acquired using a visible light commercial digital camera (Nikon 
Coolpix SQ, Japan). The initial image resolution size was 1024x768 pixels, then, reduced to 300x300 pixels. 
200 images of each water stress category were captured producing a total of 800 images from four categories. 
70% of 800 (560 images) were set for training data and the rest 30% (240 images) for validation data. Data 
testing was also required as a final model assessment taken separately at different times with a total of 240 
moss image data under different water stress conditions. 


(a) (b) (c) (d) 


Figure 1. 300x300 pixels image of moss in different water stress: (a) dry; (b) semi-dry; (c) wet; (d) soak 





CNN, one of deep learning methods, has become popular to solve computer vision-based 
agricultural problem [34]. Compare with other machine learning methods, as fundamental difference, feature 
extraction process is omitted in CNN. Instead, it can directly manage raw image to classify output by tuning 
the parameters in convolutional and pooling layers. In the classification process, deep learning architecture 
was used to classify sample moss image in four water stress stages 1.e. dry, semi-dry, wet, and soak. Structure 
of typical CNN can be seen in Figure 2. CNN structures covered image acquisition, convolutional layer, 
pooling layer, and fully connected layer. The research used the deep learning application with four types of 
pretrained CNN models (SqueezeNet [35], GoogLeNet [36], Resnet50 [37], and AlexNet [38]) provided in 
MATLAB R2020b platform. 





— Dry 
@— Semi Dry 
@— Wet 
@— Soak 

Input image Convolutional Pooling Convolutional Pooling Fully — Softmax 
Sunagoke Moss Layer Layer Layer Layer Connected 
Layer 
Feature Extraction Classification 


Figure 2. Proposed computer vision for classifying water stress in cultured moss using deep learning 
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Figure 3 (a) shows SqueezeNet architecture details with processing stages including: 1) resizing the 
image data into 227x227 pixels with depth 3 (RGB channels); 2) convolutions with stride [2 2] and padding 
[0 0 0 0]; 3) max pooling with stride [2 2] and padding [0 0 0 0]; 4) max pooling with stride [2 2] and 
padding [0 1 0 1]; 5) 2-steps fire module with stride [1 1] and padding [0 0 0 0]; 6) max pooling with stride 
[2 2] and padding [0 1 0 1]; 7) 4-steps fire module with stride [1 1] and padding [0 0 0 0]; 8) convolutions 
with 4 output, stride [1 1], and padding [0 0 0 0]; 9) average pooling; and 10) output. 

Figure 3 (b) shows GoogLeNet architecture details with processing stages including: 1) resizing the 
image data into 224x224 pixels with depth 3; 2) convolutions with stride [2 2] and padding [3 3 3 3]; 3) max 
pooling with stride [2 2] and padding [0 1 0 1]; 4) convolutions with stride [1 1] and padding [0 0 0 0]; 
5) convolutions with stride [1 1] and padding [1 1 1 1]; 6) max pooling with stride [2 2] and padding 
[0 1 0 1]; 7) 2-steps inception module; 8) max pooling with stride [2 2] and padding [0 1 0 1]; 9) 5-steps 
inception module; 10) max pooling with stride [2 2] and padding [0 1 0 1]; 11) 2-steps inception module; 12) 
average pooling; 13) fully connected layer with 4 output; 14) output. 

Figure 3 (c) shows ResNet50 architecture details consisting of 34 layers with one additional max 
pooling layer at the beginning and one average pooling layer at the end. The architecture details including: 1) 
resizing the image data into 224x224 pixels with depth 3; 2) convolutions with stride [2 2] and padding 
[3 3 3 3]; 3) batch normalization convolutions; 4) max pooling with stride [2 2] and padding [1 1 1 1]; 
5) residual block x16; 6) average pooling; 7) fully connected layer with 4 output; 8) output. 

Figure 3 (d) shows AlexNet architecture details consisting of 8 layers with 5 convolutional layers 
and 3 fully connected layers which each convolutional layer was combined with max pooling layer and one 
normalization layer in order to reduce the image resolution size and to normalize image resultant value. The 
architecture details include: 1) resizing the image data into 227x227 pixels with depth 3; 2) convolutions 
with stride [4 4] and padding [0 0 0 0]; 3) max pooling with stride [2 2] and padding [0 0 0 0]; 
4) convolutions with stride [1 1] and padding [0 0 0 0]; 5) max pooling with stride [2 2], and padding 
[0 0 0 0]; 6) 3-steps convolutions with stride [1 1] and padding [1 1 1 1]; 7) max pooling with stride [2 2] and 
padding [0 0 0 0]; 8) 3-steps fully connected layer with 4 output; 9) output. ReLu activation function was 
used in every hidden layer and softmax function was applied in the final layer to ensure the output values to 
constantly range between 0 and 1. 

ReLu activation function is given as: 


f(x) = max (0, x) (1) 


Softmax activation function is given as: 


f(x) =U (2) 


In the training process, the maximum epoch was set at the values of 20 and 30; mini-batch size was 
set at the values of 20 and 128, iteration per epoch at the values of 4 and 28, maximum iteration at the values 
of 120 and 560, the momentum of 0.9, and loss function used binary cross-entropy. Research conducted by 
Thenmozhi and Reddy [39] stated that the best learning rate of the CNN model was 0.0001; thus, the research 
set the initial learning rate at 0.0001. Moreover, the study used the following optimization techniques 
(optimizer): stochastic gradient descent with momentum (SGDm), adaptive moment estimation (Adam), and 
root mean square propagation (RMSProp) [40]. SGDm accelerated the convergence by replacing actual 
gradients with estimation, calculated from a randomly selected subset of data. Adam is an algorithm to 
optimize learning rate by combining the advantage of RMSProp and SGDm. In this research, augmentation 
data were also used to increase the data number by applying rotation (min=0; max=90) and rescaling (min=1; 
max=2). CNN program was run on a computer with the following specifications, Intel Core 13-4150 CPU 
@3.50 GHz (4 CPUs) 10 GB of RAM. Performances of 24 types of CNN models built were later evaluated 
based on the data accuracy and validation to take 5 best CNN models. The performance of those 5 best CNN 
models was tested according to the classification using confusion matrices which used data testing. The 
accuracy of each CNN model was the main parameter that determines performance by the following 
equation: 


number of correct prediction 
Accuracy = à : (3) 


total number of samples used for prediction 
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Figure 3. Schematic representation of CNN model: (a) SqueezeNet; (b) ResNet50; 
(c) AlexNet; (d) GoogLeNet 


3. RESULTS AND ANALYSIS 

Based on those 24 types of CNN models to classify water stress in moss plants, it acquired the 
accuracy value of data validation ranging from 58.33% to 87.50%. CNN model with a different number of 
layers affected prediction accuracy of water stress classification in cultured Sunagoke moss. Furthermore, the 
choice of optimization technique also influenced prediction accuracy. Table 1 shows the accuracy result of 
validation data resulted from the learning process that used deep learning architecture. Sensitivity analysis 
was carried out by tuning the optimization technique, maximum epoch, and mini-batch size for each CNN 
model i.e. SqueezeNet, GoogLeNet, ResNet50, and AlexNet. However, not all CNN models evidently 
produced performance with high accuracy value. The highest accuracy values gained by each model were 
presented in Table 1. 
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Table 1. Results and parameters used in deep learning 
Architecture Optimizer Epoch Miuni-batch Size Max Iteration Accuracy (%) Training Time (minutes) 


SqueezeNet SGDm 30 128 120 64.17 
Adam 30 128 120 70.83 29 
RMSProp 30 128 120 69.58 29 
SGDm 20 20 560 65 36 
Adam 20 20 560 72.08 35 
RMSProp 20 20 560 58.33 35 
GoogLeNet SGDm 30 128 120 67.08 54 
Adam 30 128 120 62.08 53 
RMSProp 30 128 120 63.75 53 
SGDm 20 20 560 74.17 90 
Adam 20 20 560 84.17 78 
RMSProp 20 20 560 78.75 80 
ResNet50 SGDm 30 128 120 70 848 
Adam 30 128 120 86.67 713 
RMSProp 30 128 120 87.5 701 
SGDm 20 20 560 78.75 187 
Adam 20 20 560 T4.5 191 
RMSProp 20 20 560 77.08 199 
AlexNet SGDm 30 128 120 73.75 20 
Adam 30 128 120 65.42 17 
RMSProp 30 128 120 63.75 17 
SGDm 20 20 560 63.33 43 
Adam 20 20 560 59.17 40 
RMSProp 20 20 560 60 46 


Five CNN models with highest accuracy value were reached by Resnet50 (optimizer=RMSProp, 
epoch=30, mini-batch size=128), ResNet50 (optimizer=Adam, epoch=30, mini-batch size=128), GoogLeNet 
(optimizer=Adam, epoch=20, mini-batch size=20), GoogLeNet (optimizer=RMSProp, epoch=20, mini-batch 
size=20), and ResNet50 (optimizer=SGDm, epoch=20, mini-batch size=20) with accuracy value of each 
validation data as follows, 87.50%, 86.67%, 84.17%, 78.75%, and 78.75%, respectively. The best 
performance of the 5 CNN models can be seen in Appendix. This (see Appendix) illustrates the comparison 
between classification accuracy and loss on the number of iterative learning. Based on the performance 
result, it was obvious that performance continuously improves as the iteration increases. According to the 
CNN performance graphic, it was also seen that the accuracy improved as the iteration increased, conversely, 
the loss graphic decreased as the iteration increased and the longer seen the more convergent it is. All CNN 
models have relatively the same pattern; the performance improvement run very fast in the initial epoch 
between epoch 1 to epoch 10 and the accuracy value was slowly increased followed by minor improvement 
both in data training performance and validation data. According to all results in Appendix, it can be seen 
some parameters 1.e. optimization technique, epoch number, mini-batch size, and iteration number affected 
the accuracy result of validation data. From the research result, it is seen that as epoch value increased, the 
accuracy performance did so. The bigger value of mini-batch size affected the longer a model to run, and also 
influenced memory needs. 

Figure 4 illustrates confusion matrices result by using data testing on five best CNN models such as 
GoogLeNet (optimizer=Adam, epoch=20,  mini-batch size=20); GoogLeNet (optimizer=RMSProp, 
epoch=20, mini-batch size=20); ResNet50 (optimizer-Adam, epoch-30, mini-batch size=128); Resnet50 
(optimizer=RMSProp, epoch-30, mini-batch size=128); and ResNet50 (optimizer-SGDm, epoch=20, 
mini-batch size=20). Individual classification rate for every category was described by comparing predicted 
value (abscissa) and true value (ordinate). The value distribution of the confusion matrices on the five best 
CNN models showed the trend of dry class and semi-dry class to have higher accuracy than the other two 
classes with data testing average of 88.00% for dry class and 89.66% for semi-dry class. In dry moss 
condition, it was clear that plants have roughest surface texture and browning effects on the tip of plants stalk 
due to dryness. It is in agreement with the research by Miranda [41] that level of dryness resulted in 
browning in plants. In semi-dry moss conditions, the moss has a rather rough surface texture and slightly 
browning effects on the tip of plants. The surface texture between dry and semi-dry moss was rather difficult 
to distinguish due to their almost similar rough texture and have a deeper green color. The moss plants 
showed in reverse when in wet condition, they appeared to have a relatively soft surface texture with a lighter 
green color, and almost no browning effect in this condition. Moreover, it was nearly the same in wet 
condition, the moss in soak class also had a very soft surface texture so as the result of those conditions were 
also difficult to distinguish. Accordingly, overall classification accuracy in wet and soak were lower than in 
other classes. Yet, each CNN model had different result characteristics of confusion matrices in detail. In all 
CNN models using GoogLeNet appeared that the wet and soak conditions had the highest accuracy value 
than other conditions that is 100%. In the CNN model using ResNet50, semi-dry conditions tended to have 


Classification of water stress in cultured Sunagoke moss using deep learning (Yusuf Hendrawan) 


1600 O ISSN: 1693-6930 


the highest accuracy. If it is based on the accuracy result of data testing using confusion matrices, it can be 
concluded that the highest accuracy was reached by CNN model using ResNet50 (optimizer=Adam, 
epoch=30, mini-batch size=128) with the average accuracy value of 94.15%, followed by Resnet50 
(optimizer-RMSProp, epoch=30, mini-batch size=128), ResNet50 (optimizer=SGDm, epoch=20, mini-batch 
size=20), GoogLeNet  (optimizer-RMSProp, epoch=20,  mini-batch size=20), and GoogLeNet 
(optimizer=Adam, epoch=20, mini-batch size=20) with the average accuracy value consecutively 88.75%, 
83.77%, 80.4%, and 78.33%. By the highest average accuracy of 94.15%, the CNN model produced in this 
research can be utilized effectively to classify water stress conditions in cultured Sunagoke moss. In the 
future work, it can further continue for irrigation plants control application both in the greenhouse 
(semi-closed bio-production system) and in the plant factory (fully closed bio-production system). With the 
CNN model, a rapid, real-time, and precise control system can be achieved. Thus, high-quality plant products 
and productivity can be obtained. This can also be used as a basis for developing a CNN model to detect and 
predict various kinds of stress on other various types of plants. 





True Class 




















Dry Semi-Dry Soak Welt 
Predicted Class 
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Confusion Matrix 





True Glass 





Confusion Matrix 
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Figure 4. Confusion matrices of testing data: (a) GoogLeNet (optimizer = Adam, epoch = 20, mini-batch 
size-20); (b) GoogLeNet (optimizer=RMSProp, epoch=20, mini-batch size=20); (c) ResNet50 
(optimizer-Adam, epoch=30, mini-batch size=128); (d) Resnet50 (optimizer=RMSProp, epoch=30, 
mini-batch size=128); (e) ResNet50 (optimizer-SGDm, epoch=20, mini-batch size=20). 
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4. CONCLUSION 


This research used four types of CNN model namely SqueezeNet, GoogLeNet, ResNet50, and 
AlexNet to classify four types of water stress in cultured Sunagoke moss namely dry, semi-dry, wet, and 
soak. Based on the training and validation result, it obtained accuracy value ranging from 58.33% to 87.50%. 
The highest accuracy value was obtained when using Resnet50 model with the following settings, optimizer 
parameter-RMSProp, maximum epoch=30, and mini-batch size=128. However, according to the data testing 
test result of different moss plant samples, ResNet50 model with the settings, parameter optimizer-Adam, 
maximum epoch=30, and mini-batch size=128 resulted the highest testing accuracy 94.15%. Therefore, it can 
be concluded that the CNN model resulted in this research can be effectively used to classify water stress 
condition in cultured Sunagoke moss. Then, the model can create rapid, real-time, precise, and 
non-destructive irrigation control system in plants to avoid water stress either lack or excess of water. 
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